Pressure sensitive controls

ABSTRACT

The use of pressure-sensitive controls is disclosed in which controls respond differently to different pressure applied by a stylus on a digitizer. Users interacting with displayed controls may modify the speed, increment, or other property of the control by pressing harder or softer. Devices that allow for the input of location and pressure information may use embodiments of the invention, including computers with pressure sensitive mice or digitizer tablets, PDAs, or tablet computers. Controls that may use pressure to augment their behavior include scrollbars, spinner controls, resize handles, buttons, drop down menus and so forth.

FIELD OF THE INVENTION

This invention relates generally to managing input to computers. Moreparticularly, the invention allows for acting on a user's input inaccordance with the physical pressure exerted by the user when providingthe input.

BACKGROUND OF THE INVENTION

In the field of computing, stylus-based input is growing more prevalentwith the increasingly widespread use of computing devices such aspersonal digital assistants (PDAs), tablet computers, laptop computers,and the like. The input devices used to detect contact and position by astylus shall hereinafter be referred to as digitizers. A digitizer maybe attached to or otherwise integrated with the display surface of adisplay device. The display device may further be separate from orintegral with a computing device. For example, a tablet computertypically has an integral display device with a digitizer.Alternatively, a digitizer may take the form of an external peripheralsuch as a digitizing tablet. Some types of digitizers are able totranslate a stylus contact with the digitizer into a two-dimensionalposition that corresponds to coordinates on the display surface closestto where the tip of the stylus came into contact.

Along with position, digitizers are able to detect the amount ofpressure exerted by the user when making contact with a stylus. Thisinformation may be passed to the computing device in the form of amagnitude, perhaps as an eight-bit number. Typically, however, mostoperating systems, applications, and other software ignore thisinformation, primarily interpreting contact pressure as a single click,regardless of magnitude. Notable exceptions include Adobe's Photoshop®and similar graphics programs, which use pressure on a digitizer tabletto simulate the varying strokes of a virtual paint brush.

It would be an enhancement for graphical interface users to take fulladvantage of pressure as an input throughout an operating system,applications, and/or other software. It would also be an enhancement forgraphical interface users to develop faster and more accurate control ofinterface functions using existing hardware. Further, it would be anenhancement for graphical interface users to achieve additionalintuitive functionality from existing input devices without confusingusers with additional buttons or switches.

SUMMARY OF THE INVENTION

The following presents a simplified summary in order to provide a basicunderstanding of some aspects of the invention. The summary is not anextensive overview of the invention. It is neither intended to identifykey or critical elements of the invention nor to delineate the scope ofthe invention. The following summary merely presents some concepts ofthe invention in a simplified form as a prelude to the more detaileddescription below.

A first embodiment of the invention provides a computer-implementedmethod for adjusting a displayed control. An input from a digitizer,pressure sensitive mouse, etc. is received at a location correspondingto the displayed control. The amount of pressure applied by the user isdetermined, and the displayed control is adjusted depending on theamount of pressure applied.

A second embodiment of the invention provides a computer-implementedmethod for responding to a user interaction. A tap is received upon adisplay device, and it is determined whether the tap was a hard tap. Ifa hard tap, one function is performed, but if not, a separate functionis performed.

A third embodiment of the invention provides a computer-implementedmethod for performing a function responding to a tap. The tap isreceived and location, pressure, and length of time of the tap areanalyzed to determine if the tap is a hard tap. If the pressure exceedsa certain threshold within a certain amount of time, the tap is found tobe a hard tap, and a particular function is performed. Failing the test,a different function is performed.

A fourth embodiment of the invention provides a computer-implementedmethod for interacting with displayed objects. When receiving aselection of objects via pressure enhanced input, the amount of pressureplays a role in determining the quantity/type/etc. of objects to beselected.

BRIEF DESCRIPTION OF THE DRAWINGS

A more complete understanding of the present invention and theadvantages thereof may be acquired by referring to the followingdescription in consideration of the accompanying drawings, in which likereference numbers indicate like features, and wherein:

FIG. 1 an operating environment that may be used for one or more aspectsof an illustrative embodiment of the invention.

FIG. 2 is a plan view of a digitizer display of an illustrativecomputing device.

FIG. 3 depicts a graph of pressure over time and visual feedbackprovided by an illustrative embodiment of the invention.

FIG. 4 illustrates movement of a scrollbar provided by an illustrativeembodiment of the invention.

FIG. 5 illustrates movement of a scrollbar provided by an illustrativeembodiment of the invention.

FIG. 6 illustrates incrementing a spinner control in a manner providedby an illustrative embodiment of the invention.

FIG. 7 illustrates incrementing a spinner control in a manner providedby an illustrative embodiment of the invention.

FIG. 8 illustrates resizing of an object in a manner provided by anillustrative embodiment of the invention.

FIG. 9 illustrates resizing of an object in a manner provided by anillustrative embodiment of the invention.

FIG. 10 is a flowchart for a method for adjusting a displayed controlprovided by an illustrative embodiment of the invention.

FIG. 11 illustrates selecting text in a manner provided by anillustrative embodiment of the invention.

FIG. 12 illustrates selecting drawing objects in a manner provided by anillustrative embodiment of the invention.

FIG. 13 illustrates using encounter selection to select file and folderobjects in a manner provided by an illustrative embodiment of theinvention.

FIG. 14 illustrates using an encounter selection to select file andfolder objects in a manner provided by an illustrative embodiment of theinvention.

FIG. 15 is a flowchart for a method for selecting displayed objectsprovided by an illustrative embodiment of the invention.

FIG. 16 illustrates movement of a scrollbar provided by an illustrativeembodiment of the invention.

FIG. 17 illustrates movement of a scrollbar provided by an illustrativeembodiment of the invention.

FIG. 18 illustrates selection of a file provided by an illustrativeembodiment of the invention.

FIG. 19 illustrate displaying a context sensitive menu provided by anillustrative embodiment of the invention.

FIG. 20 depicts a distance threshold for determining a type of tapprovided by an illustrative embodiment of the invention.

FIG. 21 depicts a graph of input pressure over time not resulting in ahard tap as provided by an illustrative embodiment of the invention.

FIG. 22 depicts a graph of input pressure over time resulting in a hardtap as provided by an illustrative embodiment of the invention.

FIG. 23 depicts a graph of input pressure over time not resulting in ahard tap as provided by an illustrative embodiment of the invention.

FIG. 24 depicts various input pressure thresholds over time fordetermining a type of tap as provided by an illustrative embodiment ofthe invention.

FIG. 25 is a flowchart for a method for responding to a user interactionprovided by an illustrative embodiment of the invention.

DETAILED DESCRIPTION OF ILLUSTRATIVE EMBODIMENTS

In the following description of various illustrative embodiments,reference is made to the accompanying drawings, which form a parthereof, and in which are shown by way of illustration variousembodiments in which the invention may be practiced. It is to beunderstood that other embodiments may be utilized, and structural andfunctional modifications may be made, without departing from the scopeand spirit of the present invention.

Illustrative Operating Environment

FIG. 1 illustrates an example of a suitable computing system environment100 in which the invention may be implemented. The computing systemenvironment 100 is only one example of a suitable computing environmentand is not intended to suggest any limitation as to the scope of use orfunctionality of the invention. Neither should computing environment 100be interpreted as having any dependency or requirement relating to anyone or combination of components illustrated in illustrative operatingenvironment 100.

The invention is operational with numerous other general purpose orspecial purpose computing system environments or configurations.Examples of well known computing systems, environments, and/orconfigurations that may be suitable for use with the invention include,but are not limited to, personal computers (PCs); server computers;hand-held and other portable devices such as personal digital assistants(PDAs), tablet PCs or laptop PCs; multiprocessor systems;microprocessor-based systems; set top boxes; programmable consumerelectronics; network PCs; minicomputers; mainframe computers;distributed computing environments that include any of the above systemsor devices; and the like.

The invention may be described in the general context ofcomputer-executable instructions, such as program modules, beingexecuted by a computer. Generally, program modules include routines,programs, objects, components, data structures, etc. that performparticular tasks or implement particular abstract data types. Theinvention may also be practiced in distributed computing environmentswhere tasks are performed by remote processing devices that are linkedthrough a communications network. In a distributed computingenvironment, program modules may be located in both local and remotecomputer storage media including memory storage devices.

With reference to FIG. 1, illustrative computing system environment 100includes a general purpose computing device in the form of a computer110. Components of computer 110 may include, but are not limited to, aprocessing unit 120, a system memory 130, and a system bus 121 thatcouples various system components including system memory 130 toprocessing unit 120. System bus 121 may be any of several types of busstructures including a memory bus or memory controller, a peripheralbus, and a local bus using any of a variety of bus architectures. By wayof example, and not limitation, such architectures include IndustryStandard Architecture (ISA) bus, Micro Channel Architecture (MCA) bus,Enhanced ISA (EISA) bus, Video Electronics Standards Association (VESA)local bus, Advanced Graphics Port (AGP) bus, and Peripheral ComponentInterconnect (PCI) bus, also known as Mezzanine bus.

Computer 110 typically includes a variety of computer readable media.Computer readable media can be any available media that can be accessedby computer 110 such as volatile, nonvolatile, removable, andnon-removable media. By way of example, and not limitation, computerreadable media may include computer storage media and communicationmedia. Computer storage media may include volatile, nonvolatile,removable, and non-removable media implemented in any method ortechnology for storage of information such as computer-readableinstructions, data structures, program modules or other data. Computerstorage media includes, but is not limited to, random-access memory(RAM), read-only memory (ROM), electrically-erasable programmable ROM(EEPROM), flash memory or other memory technology, compact-disc ROM(CD-ROM), digital video disc (DVD) or other optical disk storage,magnetic cassettes, magnetic tape, magnetic disk storage or othermagnetic storage devices, or any other medium which can be used to storethe desired information and which can accessed by computer 110.Communication media typically embodies computer-readable instructions,data structures, program modules or other data in a modulated datasignal such as a carrier wave or other transport mechanism and includesany information delivery media. The term “modulated data signal” means asignal that has one or more of its characteristics set or changed insuch a manner as to encode information in the signal. By way of example,and not limitation, communication media includes wired media such as awired network or direct-wired connection, and wireless media such asacoustic, radio frequency (RF) such as BLUETOOTH standard wirelesslinks, infrared and other wireless media. Combinations of the any of theabove should also be included within the scope of computer readablemedia.

The system memory 130 includes computer storage media in the form ofvolatile and/or nonvolatile memory such as ROM 131 and RAM 132. A basicinput/output system (BIOS) 133, containing the basic routines that helpto transfer information between elements within computer 110, such asduring start-up, is typically stored in ROM 131. RAM 132 typicallycontains data and/or program modules that are immediately accessible toand/or presently being operated on by processing unit 120. By way ofexample, and not limitation, FIG. 1 illustrates software includingoperating system 134, application programs 135, other program modules136, and program data 137.

Computer 110 may also include other computer storage media. By way ofexample only, FIG. 1 illustrates a hard disk drive 141 that reads fromor writes to non-removable, nonvolatile magnetic media, a magnetic diskdrive 151 that reads from or writes to a removable, nonvolatile magneticdisk 152, and an optical disk drive 155 that reads from or writes to aremovable, nonvolatile optical disk 156 such as a CD-ROM, DVD, or otheroptical media. Other computer storage media that can be used in theillustrative operating environment include, but are not limited to,magnetic tape cassettes, flash memory cards, digital video tape, solidstate RAM, solid state ROM, and the like. The hard disk drive 141 istypically connected to the system bus 121 through a non-removable memoryinterface such as interface 140, and magnetic disk drive 151 and opticaldisk drive 155 are typically connected to the system bus 121 by aremovable memory interface, such as interface 150.

The drives and their associated computer storage media discussed aboveand illustrated in FIG. 1 provide storage of computer-readableinstructions, data structures, program modules and other data forcomputer 110. In FIG. 1, for example, hard disk drive 141 is illustratedas storing an operating system 144, application programs 145, otherprogram modules 146, and program data 147. Note that these componentscan either be the same as or different from operating system 134,application programs 135, other program modules 136, and program data137, respectively. Operating system 144, application programs 145, otherprogram modules 146, and program data 147 are given different numbers inFIG. 1 to illustrate that, at a minimum, they are different copies. Auser may enter commands and information into computer 110 through inputdevices such as a keyboard 162 and pointing device 161, commonlyreferred to as a mouse, trackball or touch pad. Such pointing devicesmay provide pressure information, providing not only a location ofinput, but also the pressure exerted while clicking or touching thedevice. Other input devices (not shown) may include a microphone,joystick, game pad, satellite dish, scanner, or the like. These andother input devices are often coupled to processing unit 120 through auser input interface 160 that is coupled to system bus 121, but may beconnected by other interface and bus structures, such as a parallelport, game port, universal serial bus (USB), or IEEE 1394 serial bus(FIREWIRE). A monitor 184 or other type of display device is alsocoupled to the system bus 121 via an interface, such as a video adapter183. Video adapter 183 may comprise advanced 2D or 3D graphicscapabilities, in addition to its own specialized processor and memory.

Computer 110 may also include a digitizer 185 to allow a user to provideinput using a stylus 186. Digitizer 185 may either be integrated intomonitor 184 or another display device, or be part of a separate device,such as a digitizer pad. Computer 110 may also include other peripheraloutput devices such as speakers 189 and a printer 188, which may beconnected through an output peripheral interface 187.

Computer 110 may operate in a networked environment using logicalconnections to one or more remote computers, such as a remote computer180. Remote computer 180 may be a personal computer, a server, a router,a network PC, a peer device or other common network node, and typicallyincludes many or all of the elements described above relative tocomputer 110, although only a memory storage device 181 has beenillustrated in FIG. 1. The logical connections depicted in FIG. 1include a local area network (LAN) 171 and a wide area network (WAN)173, but may also include other networks. Such networking environmentsare commonplace in offices, enterprise-wide computer networks, intranetsand the Internet.

When used in a LAN networking environment, computer 110 is coupled tothe LAN 171 through a network interface or adapter 170. When used in aWAN networking environment, computer 110 may include a modem 172 oranother device for establishing communications over WAN 173, such as theInternet. Modem 172, which may be internal or external, may be connectedto system bus 121 via user input interface 160 or another appropriatemechanism. In a networked environment, program modules depicted relativeto computer 110, or portions thereof, may be stored remotely such as inremote storage device 181. By way of example, and not limitation, FIG. 1illustrates remote application programs 182 as residing on memory device181. It will be appreciated that the network connections shown areillustrative and other means of establishing a communications linkbetween the computers may be used.

Pressure Sensitive Input

FIG. 2 illustrates a computing device that may be used with anembodiment of the invention. The computing device here is a tabletcomputer 201, which minimally includes a computer display 202 with anintegral digitizer 203, and which may receive input via a user pressinga stylus 204 against the digitizer. Computer 110 may be embodied astablet computer 201. Although tablet computers are used throughout thisdocument as illustrative computing devices, tablet computers are onlyone among many possible computing devices that may be used to implementthe invention. Alternative embodiments may comprise, by way of example,personal computers (PCs), laptop computers, handheld computer such aspersonal digital assistants (PDAs), cell phones, home electronicequipment, or any other computing device having or being coupled to aninput device that detects input pressure, such as a digitizer or apressure-sensitive pointing device such as a pressure-sensitive mouse,pressure-sensitive trackball, or pressure-sensitive joystick. The term“pressure-sensitive” is intended to refer to a pressure-sensitive inputdevice that is capable of detecting (either directly or indirectly) anddistinguishing between different amounts of applied input pressure, asopposed to merely being able to distinguish between input versusnon-input.

Returning to FIG. 2, when stylus 204 comes in contact with the surfaceof the tablet computer's display 202, the digitizer 203 relays tocomputer 201 data representing both a two-dimensional location of thecontact, as well as an amount of the pressure applied. The amount ofpressure may be represented as a magnitude (e.g., a numeric value alonga range of numeric values), a pressure category (e.g., light, medium,heavy, etc.), or in any other manner. Digitizer 203 may continuouslyupdate this information over time as stylus 204 moves around the displaysurface and as the contact pressure increases or decreases.

Stylus 204 may be any type of stylus, such as a man-made object or ahuman body part, such as a fingertip. A man-made stylus may include, butare not limited to, a passive- or active-type pen-like stylus such as isconventionally provided with many PDAs and tablet computers.

An illustrative graph 301 of contact pressure over time is shown in FIG.3, which also depicts two forms of visual feedback provided on thedisplay 202 by illustrative embodiments of the invention. Here, stylus204 is pressed against the surface of the display 202, and thus againstthe integrated digitizer 203 as well. In this example, the contactpressure from the stylus 204 is gradually increased and then decreasedover time, as depicted in graph 301, which is typical of a tap of thestylus 204 against the digitizer 203. To give an impression of scale,the timeframe spanned by graph 301 may be just a fraction of a second,although the timeframe may be shorter or longer. The magnitude or otherrepresentation of the applied pressure may be sampled over time atvarious sample moments. For example, at a first moment in time, stylus204 has just started pressing against digitizer 203 at point 311. Theslight depression in the surface of digitizer 203 is detected asrepresentative of the applied pressure, and the amount of pressure isshown as pressure magnitude 312 along graph 301. This value may bepassed to the computer (not shown), which may modify its behaviordepending on the pressure, and also may provide feedback to a user onthe display. Pressure may be detected in any of a number of ways, suchas by directly measuring the pressure or by estimating the pressure inaccordance with other variables, such as the amount of surface areaphysically affected by the pressing of the tip of the stylus 204 againstthe digitizer 203.

Computer 110 may provide visual, tactile, and/or audible feedback to theuser in accordance with the amount of pressure applied by the stylus204. The pressure feedback provided by computer 110 may thus take manyforms, any of which may alert the user to the level of pressurecurrently being exerted. For example, visual forms of feedback mayinvolve modifying the shape, color, size, or transparency of a cursor oran underlying control presented on display 202. Alternatively, visualfeedback may take the form of a pressure gauge, which may be depicted ata fixed location on display 202. Audible forms of feedback may involveproducing a series of clicks or varying the pitch of a sound inconjunction with the changing pressure. Tactile forms of feedback mayinclude one or more intensities or frequencies of vibration of stylus204 or a housing of computer 110.

FIG. 3 depicts two illustrative visual embodiments for providingpressure feedback to a user, but there are many other ways to providethis information. Referring again to the first moment in time in graph301, the value of pressure magnitude 312 is relayed to computer 110,which may display a cursor 313 at a location corresponding to the pointof contact of stylus 204 on the display. Here, the point of arrow 317 issurrounded by cursor halo 314, the shape and/or size (e.g., diameter) ofwhich is dependent upon the current amount pressure being applied.Alternatively, computer 110 may retain the appearance of a cursor 315,and instead modify the appearance of an underlying displayed control316, which is exemplified here as a button. It should be noted thatcursors may vary, and can include hourglasses, pinwheels, insertionpoints, and so forth. In addition, a host of underlying displayedelements can be placed under the cursor and be similarly modified.

Moving rightward across FIG. 3, at a second moment in time in graph 301,stylus 204 further depresses digitizer 203 at point 321, which isrecorded as pressure magnitude 322. Point 321 may be in the samelocation on digitizer 203 as point 311 or a different location. In thefirst visual feedback example, cursor halo 324 expands or otherwisechanges size and/or shape to indicate the higher pressure to the user.Alternatively, in the second visual feedback example, the color of thebutton 326 darkens (or otherwise changes its appearance) noticeably inresponse to varying levels of applied pressure. At a third moment intime, digitizer 203 at point 331 is further depressed, registering aspressure magnitude 332. Again, point 331 may be the same location aspoints 311 and/or 321, or may be a different location, as will bediscussed further. In response to the increased pressure, cursor halo334 expands further to reflect the increase in pressure, or, in thesecond example, feedback button 336 darkens considerably. At a fourthand final moment in time, stylus 204 is beginning to lift away fromdigitizer 203 at point 341, which registers as pressure magnitude 342.In response to the decreasing pressure, cursor halo 344 begins todeflate in size, or, in the second example, button 346 begins to lightenin color or otherwise alter its appearance in a manner different fromwhen the pressure was increasing. Again, point 341 may be the samelocation as points 311, 321, and/or 331, or may be in a differentlocation.

The use of pressure feedback may not be desirable if and when computer110 is not presently using the pressure information. For example, afirst application may utilize pressure information but a secondapplication may not. Thus, where the first application is being executedand/or is in focus, pressure feedback may be provided. However, wherethe second application is being executed and/or is in focus, thenpressure feedback may not be provided. As such, conventional cursors andcontrols may be used both when pressure is and is not relevant.Feedback, such as the inclusion of a halo around a cursor, may thereforenot only provide information on the current level of pressure beingapplied, but may also relay the fact that pressure is presentlyrelevant. By viewing a cursor halo, or a color varying control, the usermay be informed that additional input can be provided through the use ofpressure. This selective providing of pressure feedback may allowcontrollable pressure input to be an easily-discoverable feature evenfor an amateur user, through simple trial and error.

Calibration may be an integral part of pressure sensitive input.Differing digitizers may translate force differently. As such, anoperating system or application which allows for the use of pressureinput may provide a calibration routine or pressure settings dialog.This may assist in standardizing input from various types of hardware.Additionally, it may allow weak or disabled individuals to vary thesensitivity of pressure measurements to accommodate their strength.

Although illustrative embodiments of the use of pressure are set forthin some detail below, other embodiments are available. For example,pressure information may be used to distinguish between which windowreceives an input. A softer contact may indicate a normal input to thecurrent window, whereas a harder contact may be directed to anotherapplication, such as a media player playing in the background. Enablingsuch uses of pressure enable a power user to work more efficiently byproviding a larger input “vocabulary” without needlessly complicatingthe interface. At the same time, average users who opt not to usepressure or users of devices without pressure sensitivity will not seetheir experience degrade. These embodiments only enhance the deviceinteraction for those wishing to use them.

Additional forms of data related to contact with an input device arepossible input modifiers. For example, an input device may detect thesurface area of contact, the temperature at the point of contact, ordampness at the point of contact. Each may be used to supplement aninput and control device behavior.

For each of the illustrative embodiments set forth below, a devicereceiving pressure information uses that information to augment theinput it receives. This information can be used in a wide variety ofways, including pressure-sensitive controls, pressure-sensitiveselection, and through the use of a variable-pressure tap, as will bediscussed in detail.

Pressure-Sensitive Controls

Controls in a graphical user interface present intuitive input metaphorsthat allow users to manipulate and maneuver through data. Such controlsare common elements found in graphical computer operating systems,individual applications, and other forms of software. Examples of knowncontrols include, but are not limited to, scrollbars, spinner controls,resize handles, checkboxes, pull down menus, and buttons. In accordancewith aspects of the present invention, controls may be augmented throughthe use of pressure data, providing users with an additional way tointeract with displayed controls. For example, by making controlspressure-sensitive, users may be presented with a more manipulable userinterface that responds faster and more accurately to their commands andthat may more closely operate in accordance with the user's intent.Value-based controls, i.e., controls that manipulate an underlyingnumeric, alphabetic, or alphanumeric value, may be better served byallowing them to be pressure-sensitive. As another example, repeatingcontrols, i.e., controls that repeat the same action when continuouslyselected, may also be better served by the addition of pressuresensitivity. A few examples of pressure sensitive controls are set forthbelow.

FIG. 4 illustrates movement of a vertical scrollbar 403 provided by anillustrative embodiment of the invention. Scrollbar 403 graphicallydisplays and updates an underlying index value indicating a currentposition within a corresponding document 404. FIG. 4 is divided into twoarbitrary “frames” 410, 411 for explanatory purposes only. The frames410, 411 show the same displayed graphical user interface as it changesover time, starting with portion 410 and ending with portion 411. Infirst frame portion 410 of FIG. 4, a cursor 401, with a cursor halo 402,indicates that the user is presently activating the down arrow button ofvertical scrollbar 403. Scrollbar 403 indicates the vertical placementof document 404 according to the vertical location of a thumb control405. The size of cursor halo 402 signifies a relatively small amount ofpressure being applied. After a period of time, in second frame 411, theuser has continued to activate the down arrow button, and the displayedportion of document 404 has scrolled down (which is actually implementedby moving document 404 up), as indicated by the new location of thumbcontrol 405. Because of the small amount pressure applied by the user,the computer scrolls scrollbar 403 relatively slowly.

FIG. 5 illustrates the same embodiment depicted in FIG. 4, and like FIG.4 is also divided into two time-lapsed frames 510, 511, where the amountof time that passes between frames 510 and 511 is the same as the amountof time that passes between frames 410 and 411. Here, in first frame510, scrollbar 403 is at the same starting point as in frame 410, butthis time, the user is pressing harder with stylus 204, as indicated bya larger cursor halo 502. As a result, in second frame 511, thumbcontrol 405 indicates that the displayed portion of document 404 hasscrolled down further than it did in FIG. 4. Because of the greateramount of pressure applied by the user when activating the down arrowbutton, computer 110 scrolls scrollbar 403 faster. Thus, in thepresented embodiment, the scrolling speed of scrollbar 403 is dependentupon the amount of pressure applied to the down arrow button ofscrollbar 403. The same may apply to the up arrow button of scrollbar403, as well as the left and right arrow buttons of a horizontalscrollbar. The scrolling speed of scrollbar 403 may have anyrelationship to the applied pressure, such as a linear relationship or anonlinear relationship.

FIG. 6 illustrates incrementing a spinner control 601 provided by anillustrative embodiment of the invention. Spinner control 601 has anentry field 602, where a user can view or change a numeric value, and uparrow button 603 and down arrow button 604, the selection of whichrespectively increments or decrements the value in the entry field. Asin FIGS. 4 and 5, FIG. 6 is also divided into frames 610, 611, 612,which show the same graphical user interface changing over time fromleft to right in that figure. In first frame 610, entry field 602contains an initial value 0. Once a user activates up arrow button 603,the value in entry field 602 begins to increment. In second frame 611,this activation is indicated by arrow cursor 605 with cursor halo 606.As indicated by the size of cursor halo 606, light pressure is beingused to manipulate the control, and as such, the value in entry field602 has increased at a rate and/or increment (e.g., incrementing by 1)that depends upon the applied pressure. In third frame 612, the increasecontinues at a rate and/or increment that depend upon the appliedpressure. In FIG. 6, it is assumed that the same amount of time haselapsed between frame 610 and 611 as between frames 611 and 612, and forsimplicity that the applied pressure has remained constant throughoutframes 610-612.

FIG. 7 illustrates the same embodiment depicted in FIG. 6, and like FIG.6 is also divided into three time-lapsed frames 710, 711, 712, where theamount of time that passes between frames 710 and 711 is the same as theamount of time that passes between frames 610 and 611, and the amount oftime that passes between frames 711 and 712 is the same as the amount oftime that passes between frames 611 and 612. In first frame 710, spinnercontrol 601 has been reset to the same initial value as in frame 610.However, in second frame 711, when greater pressure is applied toactivate up arrow button 603, the value in entry field 602 is updatedusing a larger increment and/or at a greater rate than in FIG. 6. Thehigher pressure is indicated by the larger cursor halo 706. In third andfinal frame 712, the higher pressure continues updating the value usingthe larger increment and/or at a greater rate. Thus, the value of aspinner control may increase or decrease by an increment and/or at arate that depends linearly or nonlinearly upon the applied pressure.

Additionally or alternatively, with value-based controls, such as thespinner and scrollbar controls, pressure sensitivity may be used tocontrol the rate of adjustment of a value or index. Although in theabove examples, the pressure applied remained constant, that does notneed to be the case. Indeed, a user may continuously and dynamicallyincrease and decrease the pressure applied when providing an input inorder to affect the operation of the control. As the pressure changes,so does the affect on the control, such as the rate of change. Moreover,the value need not be numeric, and could be chosen from any defined set,such as a set of characters, words, colors, patterns, and so forth,depending on the particular need. For example, a color selection can bemade by using pressure to speed and slow the rate of change among aselection of colors and/or to control the intensity or other property ofa color. Another embodiment may involve selecting a state in a pull downmenu and using pressure to speed and slow the movement of the list.

Pressure sensitivity may further be used to constrain the change of avalue, as depicted for example in FIGS. 8 and 9. FIG. 8 illustratesresizing a drawing object 801 using an illustrative embodiment of theinvention. The context here may be a drawing application or otherprogram that allows users to resize objects. In FIG. 8, drawing object801 is being resized by a user. Arrow cursor 803 having cursor halo 804is positioned over resize handle 802 at one corner of object 801. Lightpressure is applied by the user, as indicated by the small cursor halo804. As such, the application allows the user to resize following asmooth, freeform path. At the end of the path, resize handle 802 hasbeen repositioned at location 805.

FIG. 9 illustrates the same embodiment of the invention depicted in FIG.8. The difference here is that the user is applying more pressure, asindicated by larger cursor halo 904. Because of this greater pressure,the application may cause the movement of resize handle 802 to benon-smooth, such as by constraining it to fixed positions on apredetermined grid, for example. Here, resize handle 802 is relocated atlocation 905. Along the way, resize handle 802 is limited to locationsalong a fixed grid, which may or may not be visible to the user.Alternatively, the amount of pressure applied may affect the gridincrement by which object 801 can be resized. In addition, an operatingsystem or application may reverse the relationship, and constrain resizemovement to a grid when less pressure is applied and allowing freeformresizing only when more pressure is applied.

The pressure-sensitive control (i.e., resize handle 802) disclosed inFIGS. 8 and 9 is not necessarily limited to resizing objects. It mayapply to other controls such as for moving objects, for example. It mayalso be used for resizing or repositioning windows in an operatingsystem or application. Ultimately, the technique can be used in anyinstance where values defining an object's screen position, shape, size,and/or any other property are being modified. Furthermore, other formsof pressure-sensitive controls may be implemented. For example, a buttonor other control that performs an action when activated and that repeatsthe action when continuously activated may be pressure-sensitive. Such arepeating control may repeat the action faster when greater pressure isapplied to the input, and likewise slow down the rate of repetition whenless pressure is applied. An additional implementation involves the useof harder pressure to vary the behavior of a drag operation. Forexample, dragging a drawing object while pressing harder may cause theobject to paste in repeated fashion, similar to a rubber stamp effect;the harder the drag, the more frequent the repeating stamps.

FIG. 10 illustrates a method for adjusting a displayed control providedby an illustrative embodiment of the invention. In step 1001, computer110 displays a pressure sensitive control on display 202, for example ascrollbar, resize handle, button, spinner control, and so forth. When auser directs input to the control, coupled with pressure applied by theuser with stylus 204, computer 110 receives the input, as in step 1002.At step 1003, computer 110 determines the amount of pressure applied bythe user. Alternatively, computer 110 may, at this point, compare thepressure applied to an amount applied previously in order to determineits relative magnitude. At step 1004, computer 110 moves or modifies therespective control (including adjusting any underlying value) inaccordance with the amount of pressure applied. A greater amount ofpressure from the user may cause adjustments to the control carried outin a first manner, perhaps by speeding up or using larger increments,whereas less pressure may cause adjustments to occur in a second manner,such as slower adjustments, or even cause the control to behave as ifnot pressure-sensitive. At decision 1005, computer 110 determineswhether the user input continues, and if so, performs steps 1002-1005again. Otherwise, the method terminates and/or awaits another userinput.

Pressure Based Selection

Item selection is a common activity for users of graphical computeroperating systems, applications, and other software, and activity thatcan benefit from pressure sensitivity. Selecting choices from a list,words in a document, files in a folder, etc. are tasks with which mostusers are familiar. Pressure sensitivity enhances the process of itemselection, for example when the use of double or triple mouse clicks areneeded to broaden a selection but no mouse is available, such as with atablet computer. For example, by pressing harder, a user signals that hewants to select a larger number of items. A user need not attempt anotoriously difficult double or triple click with a stylus on a tabletcomputer.

FIG. 11 illustrates selecting text in a word processing application in amanner provided by an illustrative embodiment of the invention. As inseveral of the previous figures, FIG. 11 is divided into three frames1100, 1110, 1120. Frames 1100, 1110, and 1120 illustrate how a graphicaluser interface may react to different applied pressures. In first frame1100, a word 1102 in paragraph 1101 is being selected by cursor 1103having cursor halo 1104. A user applies a small amount of pressure toselect word 1101. Alternatively, with a small amount of pressure, theuser may simply place an insertion point in the middle of word 1102. Asthe user presses harder, she begins to select more text, as shown insecond frame 1110, wherein more of paragraph 1101, such as a line orsentence, is selected in the form of sentence 1111. The greater pressureis reflected as visual feedback in the form of larger cursor halo 1114.In third frame 1120, the user presses harder still, reflected in largercursor halo 1124, which selects the entire paragraph 1101.Alternatively, pressing even harder may select a whole document ordocument section. Conversely, decreasing the selection pressure maydeselect a paragraph and select only a word or sentence again. Thus,different levels of applied pressure on a displayed document may causedifferent amounts of the document to be selected.

FIG. 12 illustrates selecting drawing objects in a drawing softwareprogram in a manner provided by an illustrative embodiment of theinvention. As in FIG. 11, FIG. 12 is divided into three frames 1200,1210, 1220 that illustrate how a graphical user interface may react todifferent applied pressures. In first frame 1200, drawing object 1201 isselected by a user using a small amount of pressure, as indicated bycursor 1202 having a small cursor halo 1203. Selected object 1201 may besurrounded by a selection tool embodied as a selection border 1204. Eachdrawing object within the selection border 1204 is part of theselection. In this example, the size of the selection border 1204 (i.e.,the area encompassed by the selection border 1204) depends upon theamount of applied pressure. As indicated by the size of cursor halo1203, the user is pressing lightly in order to select the objectcurrently under cursor 1202. As the user presses harder, in second frame1210, selection border 1204 grows in accordance with the higher appliedpressure, and in this case grows sufficiently large so as to encompassmore objects, including, for example, object 1215. Cursor halo 1213reflects the increasing pressure applied by the user. As the userpresses harder still, in third and final frame 1220, selection border1204 grows larger still, encompassing more drawing objects in this caseincluding, for example, object 1226. Selection border 1204 may beconstrained to grow to encompass only objects connected or adjacent tothe originally-selected object, or may be configured to also encompassobjects not connected or adjacent to the originally-selected object. Asbefore, reducing the pressure applied may return the selection to asmaller number of objects.

Alternative forms of item selection are known which may be enhancedthrough the use of pressure based selection. For example, U.S. PatentApplication Publication No. 20040021701 A1, entitled “Freeform EncounterSelection Tool,” which is hereby incorporated by reference as to itsdisclosure of an encounter selection tool, discloses a freeformencounter selection tool for a computer system with a graphical userinterface that allows a user to draw a freeform selection path so as toselect one or more items along or near the path. As the user drags apointing device such as a stylus or a mouse, a freeform selection pathis created so that the encounter selection tool selects graphical itemsthat are encountered.

FIG. 13 illustrates using an encounter selection tool to select file andfolder objects by an illustrative embodiment of the invention. Here, asubset of a collection 1301 of files and folders is selected by draggingcursor 1302, having cursor halo 1303, from start point 1304 to end point1306. Folders and files encountered along the path of the cursor, forexample folder 1305, are selected and highlighted as shown. The user maythen perform a common function among all of the files, such as throwingthem away. As can be seen from the cursor halo, the user is onlypressing lightly when he uses the encounter select tool. This leads to arelatively narrow selection path. In this embodiment, the lighter thepressure, the narrower the selection path, and thus in general the fewerthe number of objects selected along the selection path.

FIG. 14 illustrates the same collection 1301 of files and folderspresented in FIG. 13. Here, however, the user presses harder whilemoving cursor 1302 from start point 1304 to end point 1306, as reflectedin larger cursor halo 1403. The result of the increased pressure is thata wider selection path is created and a larger number of objects isselected, including, for example, document 1405. Although thisembodiment has been discussed with regard to a freeform selection paththat follows movement of the cursor 1306, the selection path may takeother forms such as a linear path that extends between the start point1304 and the end point 1306, regardless of the path the cursor 1306takes between the start and end points 1304, 1306.

FIG. 15 illustrates one method for selecting displayed objects providedby an illustrative embodiment of the invention. In step 1501, acollection of selectable items such as characters, drawing objects, fileicons, etc. is displayed. A user then selects at least one of the itemswith stylus 204, and the stylus input is received in step 1502. At step1503, the amount of pressure applied by the user with the stylus 204 isdetermined so that, in step 1504, the identities of which items areselected can be modified. For example, when a user presses harder duringa selection operation, the selection path widens, and so more of theitems may be selected. If there is more input from the user, at decision1505, the steps continue. Otherwise, the method terminates normally orawaits further user input.

The pressure-based selection embodiments presented above are onlyrepresentative, and other forms of selection may be enhanced through theinclusion of pressure information. For example, a lasso selection tool,which is familiar from its use in graphics editing software, may beenhanced with pressure information. In such software, lasso selectionallows a user to encircle in freeform fashion a graphic of interest andcut or copy it. A user, by pressing harder while encircling a selectedgraphic of interest, may control the amount of feathering used to softenthe edges of the graphic of interest when it is cut or copied, orwhether to select objects only partially within the lasso (e.g., lowerpressure does not select such objects while higher pressure does selectsuch objects). Additionally, pressure based selection may allow for aselection zoom. For example, while pressing harder on a pixel to selectit, the screen may zoom in further allowing greater detail to bedisplayed. The user can then achieve more precision while selectingneighboring pixels.

Hard Tap

Tapping a digitizer is a common form of user interaction with acomputing device, such as a tablet computer, which can be enhanced byexploiting pressure information available from the digitizer. The termtap includes the contact and removal of a stylus, such as a pen, finger,or any other pointing implement, with the surface of the digitizer.Typically, a tap is interpreted as equivalent to a mouse click,regardless of how much force was used to impact the digitizer. Pressureinformation can be used, however, to distinguish normal taps from hardtaps, enabling new sources of user input. For example, a tap with anapplied pressure within a given first pressure range may be considered anormal tap, whereas a tap with an applied pressure within a given highersecond pressure range may be considered a hard tap. Any number ofpressure ranges may be defined with an associated tap type. A normal tapmay, for example, be interpreted as a simple click, and a hard tap may,for example, be used to trigger additional functionality. For example, ahard tap may be interpreted as a double-tap (notoriously difficult ondigitizer displays), or as a right click, or as a trigger for anon-screen keyboard for tapping out words and sentences, or as a requestto launch an application, or as a middle click (on a three buttonmouse), or as a scroll wheel click, and so forth.

FIGS. 16 and 17 illustrate one embodiment of hard taps. As in someprevious figures, FIGS. 16 and 17 are each divided into two arbitraryframes 1610, 1611, 1710, 1711 that show how a displayed graphical userinterface is affected by different types of taps. FIG. 16 depictsmovement of a scrollbar under conditions of a normal tap provided by anillustrative embodiment of the invention. Here, in first frame 1610, auser taps on scrollbar 1603, the tap being indicated by the temporaryplacement of cursor 1601 and starburst halo 1602. The starburst halo1602 may signify to the user that a tap is being received as opposed toa press-and-hold. The small starburst halo 1602 indicates, in this case,that the tap was not very hard. In second frame 1611, the results of thetap are viewable. Document 1604 has scrolled down one page, and thethumb control 1605 has shifted down.

First frame 1710 of FIG. 17 illustrates the same starting position ofdocument 1604 and thumb control 1605. A user taps the same location onscrollbar 1603 as before, but this time taps harder. The starburst halo1702 that temporarily appears is larger, indicating that a harder tapwas registered than in FIG. 16. Rather than page down as before, thehard tap in this case triggers a different function. As can be seen insecond frame 1711, thumb control 1605 jumps directly to the location ofthe harder tap. This can be useful for a user who wants to go directlyto a portion of document 1604 without waiting for the scrollbar to pagedown.

FIGS. 18 and 19 illustrate a second embodiment of hard and normal taps.As before, the figures are each divided into two arbitrary frames 1810,1811, 1910, 1911 that show how the effect of different types of taps. Infirst frame 1810 of FIG. 18, a file 1801 receives a single normal tap,as signified by cursor 1802 with starburst halo 1803. As a result of thelower pressure applied, in second frame 1811, computer 110 interpretsthe normal tap as a left click and file 1804 is highlighted. The firstframe 1910 of FIG. 19 differs from that of FIG. 18 in that file 1801receives a hard tap, as signified by larger starburst halo 1903. As aresult, computer 110 performs a separate action in second frame 1911,treating the hard tap as a right click, and displaying the contextsensitive menu rather than selecting the file.

As stated, the variable-pressure tap embodiments described above areonly a few of the uses of such an enhancement. In addition, there aremany embodiments for providing feedback about the type of tap beingreceived. The starburst halo described above is merely demonstrative.For example, other forms of visual feedback include changing the coloror transparency of the point of impact rather than changing the cursor.In addition, audio feedback may distinguish a hard tap from another tapor input. The existence of a particular sound, or the volume of a sound,or a particular pitch may provide the feedback needed by a user todistinguish tap types. The methods for evaluating a hard tap anddistinguishing hard taps from other forms of input are set forth in somedetail below.

Distinguishing a hard tap from other user inputs on a digitizer mayinvolve determining whether the tip of stylus 204 maintains asubstantially constant position on the surface of digitizer 203. A tapthat moves across the surface of digitizer 203 is more likely anintended drag operation rather than a tap, and so a distance thresholdmay be used to ensure that the point of contact does not move too far.Such a threshold is depicted in FIG. 20, which provides an illustrativelayout for a distance threshold on an X-Y coordinate plane defininglocations on the surface of digitizer 203. Here, the initial point ofcontact is represented by the black unit square in the middle of thefigure. Unit squares in FIG. 18 are shaded for explanatory purposesonly, and are not necessarily displayed as shown, if at all, to theuser. Each unit square (or other shaped area) may represent a pixel onthe underlying display and/or a minimum resolvable area that may besensed by digitizer 203, or any other unit of area, whether arbitrary orwhether defined in accordance with properties of display 202 and/ordigitizer 203. The immediately adjacent squares or pixels (cross-hatchedin FIG. 18) form a distance threshold. By way of example, stylus 204 mayinitially impact the pressure-sensitive surface of digitizer 203 at theblack square, and stylus 204 may immediately thereafter slide just alittle bit. However, if, over time, stylus 204 moves outside thedistance threshold (i.e., in this example, moves outside thecross-hatched unit squares), then a hard tap is not registered bycomputer 110. This distance threshold is configurable. If a user has adifficult time holding a pointing implement steady, she may be able toadjust the distance threshold, such as by increasing the distancethreshold to include a larger range of acceptable contact points.

If stylus 204 stays inside the distance threshold for an appropriateperiod of time, computer 110 still may not register a hard tap. Ingeneral, computer 110 may determine whether a tap input by stylus 204 isa hard tap depending upon applied pressure, hold time, and/or slidedistance of stylus 204. In this example, for the tap to be a hard tap,the tap input must reach an appropriate pressure threshold within aparticular time threshold. FIG. 21 depicts a graph 2101 of inputpressure over time not resulting in a hard tap as provided by anillustrative embodiment of the invention. Here, the tap has stayedwithin the appropriate distance threshold, but by the time the contactpasses time threshold 2102 (perhaps set at ¼ of a second or anotheramount of time) at point 2104, the magnitude of pressure has not passedpressure threshold 2103. As a result, computer 110 may interpret theinput as a normal tap or some other type of input.

FIG. 22 depicts a graph 2201 of input pressure over time resulting in ahard tap as provided by an illustrative embodiment of the invention.Here, the contact being graphed has surpassed a pressure threshold 2203at a point 2204 before reaching a time threshold 2202. This maytherefore be registered as a hard tap, and the operating system orapplication can process it as such. If the operating system orapplication does not take advantage of hard taps, then the tap can beinterpreted as a normal tap.

As another example, FIG. 23 depicts a graph 2301 of input pressure overtime in which the tap input does not register as a hard tap as providedby an illustrative embodiment of the invention. In this example, thecontact with the surface of digitizer 203 eventually exceeds a pressurethreshold 2303, but not within a time threshold 2302. When the time ofcontact exceeds time threshold 2302 at a point 2304, similar to theexample in FIG. 21, the input can be passed on as something other than ahard tap, such as a normal tap.

It should be noted that the time and pressure thresholds used to detecta hard tap may be user- or software-configurable. Individual users canperfect their successful use of the hard tap by adjusting the pressurethreshold and time threshold. For example, a particular user may not beable to achieve the pressure magnitude needed, and can adjust thepressure and/or time thresholds as desired, such as to adjust thepressure threshold lower to allow for “softer” hard taps. Pressureand/or time thresholds may further be automatically adjusted through acalibration routine. For example, computer 110, via a user interface,may request that the user execute what the user considers to be a normaltap as well as a hard tap. Computer 100 may measure the pressure andtime properties of the user's input and automatically determineappropriate time and/or pressure thresholds in accordance with thoseproperties.

Alternative embodiments of the hard tap may allow for additionalvariations of the tap using multiple time, distance, and/or pressurethresholds. For example, FIG. 24 depicts a set of pressure ranges overtime as provided by an illustrative embodiment of the invention. Here,taps not exceeding a pressure threshold 2401 within time threshold 2403will be considered normal taps. Taps exceeding a pressure threshold2401, but not exceeding a higher pressure threshold 2402 within timethreshold 2403 will be interpreted as medium taps. And, taps exceedingpressure threshold 2402 will be interpreted as hard taps. Medium tapsmay be useful in some interface environments. Individual applicationsmay provide different thresholds depending on their need, overridingthresholds set by the operating system.

FIG. 25 us a flowchart for a method for responding to a user interactionprovided by an illustrative embodiment of the invention. In step 2501,computer 110 receives a stylus input upon digitizer 203, providing dataabout the tap including location and pressure over time. Given thisinformation, at decision 2502, computer 110 determines whether thecontact location has moved within a predetermined threshold distancefrom its initial point. If not, then no tap is found, as the input maybe the beginning of a drag operation, or possibly handwriting.Alternatively, computer 110 may decide that such an input is a normaltap. If stylus 204 remains within the threshold distance during theinput, then at decision 2503, computer 110 determines whether theapplied pressure exceeds a predetermined threshold pressure within apredetermined time threshold. If not, then a normal tap is detected andthe appropriate function is executed at step 2505. If the thresholdpressure was reached within the time threshold, then a hard tap isdetected and the appropriate function is executed at step 2504.

While the invention has been described with respect to specific examplesincluding presently preferred modes of carrying out the invention, thoseskilled in the art will appreciate that there are numerous variationsand permutations of the above described devices and techniques that fallwithin the spirit and scope of the invention as set forth in theappended claims. A claim element should not be interpreted as being inmeans-plus-function format unless the phrase “means for”, “step for”, or“steps for” is included in that element. Also, numerically-labeled stepsin method claims are for labeling purposes only and should not beinterpreted as requiring a particular ordering of steps.

1. A computer-implemented method for adjusting a displayed control,comprising the steps of: (1) receiving an input upon a display surfaceof a display device at a location corresponding to the displayedcontrol; (2) determining an amount of pressure applied by the input; (3)responsive to a first amount of pressure, the displayed controlresponding in a first manner; and (4) responsive to a second greateramount of pressure, the displayed control responding in a second manner,wherein the second manner is different than the first manner, whereinstep (1) comprises receiving the input upon the display surface at alocation corresponding to a resize handle associated with an object, andwherein step (3) comprises resizing the object in a smooth manner inresponse to the first amount of pressure, and wherein step (4) comprisesresizing the object in a constrained manner in response to the secondamount of pressure.
 2. The method of claim 1, wherein step (4) comprisesresizing the object so as to be constrained to a grid and step (3)comprises resizing the object so as to not be constrained to the grid.3. The method of claim 1, wherein the object is a window containinganother displayed object.